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^>! Abstract 



We propose and analyze a broad family of games played by resource-constrained players, which are 
characterized by the following central features: 1) each user has a multi-dimensional action space, subject 



L^ . to a single sum resource constraint; 2) each user's utility in a particular dimension depends on an additive 

coupling between the user's action in the same dimension and the actions of the other users; and 3) each 
user's total utility is the sum of the utilities obtained in each dimension. Familiar examples of such multi- 
user environments in communication systems include power control over frequency-selective Gaussian 
interference channels and flow control in Jackson networks. In settings where users cannot exchange 
messages in real-time, we study how users can adjust their actions based on their local observations. We 
derive sufficient conditions under which a unique Nash equilibrium exists and the best-response algorithm 

o 
in 

in real-time, we focus on user choices that optimize the overall utility. We provide the convergence 



converges globally and linearly to the Nash equilibrium. In settings where users can exchange messages 
in real-time, we focus on user choices that optimize the overall utility. We provide the 
conditions of two distributed action update mechanisms, gradient play and Jacobi update. 



^ ■ Index Terms 



Game theory, multi-user communications, Nash equilibrium, best-response dynamics, Gradient play, 
Jacobi update, pricing mechanism. 

I. Introduction 

Game theory provides a formal framework for describing and analyzing the interactions of multiple 
decision-makers. Recently, there has been a surge in research activities that adopt game theoretic tools 
to investigate a wide range of modern communications and networking problems, such as flow and 
congestion control, network routing, load balancing, power control, peer-to-peer content sharing, etc [1 ]- 
0. In resource-constrained communication networks, a user's utility is usually not only affected by 



its own action but also by the actions taken by all the other users sharing the same resources. Due to 
the mutual coupling among users, the performance optimization of multi-user communication systems is 
challenging. Depending on the characteristics of different applications, numerous game-theoretical models 
and solution concepts have been proposed to characterize the multi-user interactions and optimize the 
users' decisions in communication networks. A variety of game theoretic solutions have been developed 
to characterize the resulting performance of the multi-user interactions, including Nash equilibrium (NE) 
and Pareto optimality |6). 

The majority of the existing game theoretic research works in communication networking applications 
usually depend on the specific structures and inter-user coupling of their action sets and utility functions. 
By considering or even architecting these specific structures, the associated games become analytically 
tractable and possess various important convergence properties. For instance, if users cannot exchange 
messages with each other and choose to individually maximize their utilities, to show the existence of and 
the convergence to a pure NE, several well-investigated classes of game models, such as concave games, 
supermodular games, and potential games, have been extensively applied in various communication 
scenarios 0- lfT3l . When real-time information exchange is possible, various mechanisms have also 
been proposed to enable collaborative users to jointly improve their performance and find the optimum 
joint policy. A well-known example is the framework of network utility maximization (NUM) started by 
Kelly etc. lfl4l ITT31 . which has recently been widely adopted to analyze the problems related to fairness 
and efficiency in communication networks. Moreover, various distributed resource allocation algorithms 
have been developed to implement the NUM framework in an informationally-decentralized manner. 
In particular, if a convex NUM problem can be decomposed into several subproblems by introducing 
Lagrange multipliers associated with different resource constraints, the global optimum can be computed 
using distributed algorithms by deploying message passing mechanisms [16]. 

Power control is one of the first few communication problems in which researchers started to apply 
game theoretic tools to formalize the multi-user interaction and characterize its properties. An interesting 
and important topic that has been extensively investigated recently is how to optimize multiple devices' 
power allocation when sharing a common frequency-selective interference channel. In ifTTl . Yu et. al. 
first defined such a power control game from a game-theoretic perspective, proposed a best-response 
algorithm in which all users iteratively update their power allocations using the water-filling solution, 
and proved several sufficient conditions under which the algorithm globally converges to a unique pure 
NE. Many follow-up papers further establish various sufficient convergence conditions with or without 
real-time information exchange for power control in communication networks [18]- 1221 . The purpose of 



this paper is to introduce and analyze a general framework that abstracts the common characteristics of 
this family of multi-user interaction scenarios, which includes, but is not limited to, the power control 
scenario. In particular, the main contributions of this paper are as follows. 

First of all, we define the class of Additively Coupled Sum Constrained Games (ACSCG), which 
captures and characterizes the key features of several communication and networking applications. In 
particular, the central features of ACSCG are: 1) each user has a multi-dimensional strategy that is 
subject to a single sum resource constraint; 2) each user's payoff in each dimension is impacted by an 
additive combination of its own action in the same dimension and a function of the other users' actions; 
3) users' utilities are separable across different dimensions and each user's total utility is the sum of the 
utilities obtained within each dimension. 

Second, based on the feasibility of real-time information exchange, we provide the convergence 
conditions of various generic distributed algorithms in different scenarios. When no message exchanges 
between users are possible and every user maximizes its own utility, it is essential to determine whether a 
NE exist and if yes, how to achieve such an equilibrium. In ACSCG, a pure NE exists in ACSCG because 
ACSCG belongs to concave games Q. Our key contribution in this context is that we investigate 
the uniqueness of pure NE and consider the best response dynamics to compute the NE. We explore the 
properties of the additive coupling among users given the sum constraint and provide several sufficient 
conditions under which best response dynamics converges linearljLj to the unique NE, for any set of 
feasible initialization with either sequential or parallel updates. We also explain the relationship between 
our results and the conditions previously developed in the game theory literature Q 11241 . When users 
can collaboratively exchange messages with each other in real-time, we present the sufficient convergence 
conditions of two alternative distributed pricing algorithms, including gradient play and Jacobi update, to 
coordinate users' action and improve the overall system efficiency. The proposed convergence conditions 
generalize the results that have been previously obtained in ifTTl - IT221 for the multi-user power control 
problem and they are immediately applicable to other multi-user applications in communication networks 
that fulfill the requirements of ACSCG. 

The rest of this paper is organized as follows. Section II defines the model of ACSCG. For ACSCG 
models, Sections III and IV present several distributed algorithms without and with real-time information 
exchanges, respectively, and provide sufficient conditions that guarantee the convergence of the proposed 

'A sequence x^ ' with limit x* is linearly convergent if there exists a constant c £ (0, 1) such that \x^ k '~ x*\ < c|a;' ' — x*\ 
for k sufficiently large 1231 . 



algorithms. Section V presents the numerical examples and conclusions are drawn in Section VI. 

II. Game Model 

In this section, we introduce some basic definitions from the theory of strategic games to characterize 
the multi-user interaction, define the model of ACSCG, and present some illustrative examples for the 
class of ACSCG. 

A. Strategic Games, Nash equilibrium, and Pareto Optimality 

A strategic game is a suitable model for the analysis of a game where all users act independently and 
simultaneously according to their own self-interests and with no or limited a priori knowledge of the other 
users' strategies. This can be formally defined as a tuple T = (J\f, A, u). In particular, J\f = {1,2, ... , N} 
is the set of decision makers. Define A to be the joint action set A = x n€ j^A n , with A n C TZ K being 
the action set available for user n. The vector utility function u = x n ^N u n is a mapping from the 
individual users' joint action set to real numbers, i.e. u : A — > 1Z N . In particular, u„(a) : „4 — ^ 7£ is the 
utility of the nth user that generally depends on the strategies a = (a n ,a_ n ) of all users, where a n E A n 
denotes a feasible action of user n, and a_ n = x m ^ n a m is a vector of the actions of all users except 
n. We also denote by A- n = x m ^ n A m the joint action set of all users except n. To capture the multi- 
user performance tradeoff, the utility region is defined as 14 = {(ui(a), . . . , ujy-(a))| 3 a E A}. Various 
game theoretic solutions, such as NE and Pareto optimality, were developed to characterize the resulting 
performance |6]. Significant research efforts have been devoted in the literature to constructing operational 
algorithms in order to achieve NE and Pareto optimality in various games with special structures of action 
set A n and utility function u n . 

1) Nash equilibrium: definition, existence, and convergence: To avoid the overhead associated with 
exchanging information in real-time, network designers may prefer fully decentralized solutions in which 
the participating users simply compete against other users by choosing actions a n E A n to selfishly 
maximize their individual utility functions u n (a n ,a_ n ), given the actions a__„ E A- n . Most of these 
approaches focus on investigating the existence and properties of NE. NE is defined to be an action 
profile (a*, a£, • • • , &* N ) with the property that for every player, it satisfies u n (a* , af_ n ) > u n (a n , a!_ n ) for 
all a n E A n , i.e. given the other users' actions, no user can increase its utility alone by changing its action. 
For an extensive discussion of the methodologies studying the existence, uniqueness, and convergence 
of various equilibria in communication networks, we refer the readers to [25]. Many of the well-known 
results rely on specific structural properties of action set A and utility function u in the investigated 



multi-user interactions. For example, to establish the existence of and convergence to a pure NE, we can 
examine whether A and u satisfy the conditions of concave games, supermodular game, potential game, 
etc. Specifically, to apply the existence result of a pure NE in concave games (6l Q, we need to check 
the following conditions: i) each player's action set A n is convex and compact; and ii) the utility function 
u n (a n ,a- n ) is continuous in a and quasi-concavdj in a n for any fixed a_ n . As additional examples of 
games that guarantee the convergence to NE, it is well-known that, in supermodular games JH (TO] and 
potential games lfl2l Ifl3l . the best response dynamics can be used to search for a pure NE. Suppose 
that utility function u n is twice continuously differentiable, Vn G J\f. If A n is a compact subset of 1Z (or 
more generally A n is a nonempty and compact sublatticdj of 1Z K ), Vn G J\f, establishing that game T is 
a supermodular game is equivalent to showing that u n satisfies 

V(m,n) £M 2 ,m^n ^" > 0. (1) 

oa n oa m 

If action set A in game T is an interval of real numbers, we can show that game T is a potential game 

by verifying 

V(m, n) £ AA 2 , m ± n, ^ ~ Um) = 0. (2) 

2) Pareto optimality and network utility maximization: It is important to note that operating at a Nash 
equilibrium will generally limit the performance of the user itself as well as that of the entire network, 
because the available network resources are not always effectively exploited due to the conflicts of interest 
occurring among users. As opposed to the NE-based approaches, there exists a large body of literature 
that focuses on studying how users can jointly improve the system performance by optimizing a certain 
common objective function /(ui(a),U2(a), . . . , UAr(a)). This function represents the fairness rule based 
on which the system-wide resource allocation is performed. Different objective functions, e.g. sum utility 
maximization in which /(iti(a), u-zfa), ... ,un (a)) = En=i u "( a )' can P rov ide reasonable allocation 
outcomes by jointly considering fairness and efficiency. A profile of actions is Pareto optimal if there is 
no other profile of actions that makes every user at least as well off and at least one user strictly better 
off. 

The majority of these approaches focus on studying how to efficiently or distributedly find the optimum 
joint policy. There exists a large body of literature that investigates how to compute Pareto optimal 



2 A real-valued function / is quasi-concave if dom/ is convex and {x £ dom/|/(x) > a} is convex for all a 
3 A real _K"-dimensional set V is a sublattice of 1Z K ii 
and the component-wise maximum, a V b, are also in V 



3 A real AT-dimensional set V is a sublattice of 1Z. K if for any two elements a, b £ V, the component-wise minimum, a A 6, 



solutions in large-scale networks where centralized solutions are infeasible. Numerous convergence results 
have been obtained for various generic distributed algorithms. An important example is the NUM 
framework that develops distributed algorithms to solve network resource allocation problems [15]. The 
majority of the results in the existing NUM literature are based on convex optimization theory, in which the 
investigated problems share the following structures: the objective function /(iti (a), 112(a), • ■ ■ , ujv(a)) is 
convexQ, inequality resource constraint functions are convex, and equality resource constraint functions are 
affine. It is well-known that, for convex optimization problems, users can collaboratively exchange price 
signals that reflect the "cost" for consuming the constrained resources and the Pareto optimal allocation 
that maximizes the network utility can be determined in a fully distributed manner ifToll . 

Summarizing, these general structural results without and with real-time message exchange turn out 
to be very useful when analyzing various multi-user interactions in communication networks. Numerous 
existing works are devoted to constructing or shaping the multi-user coupling such that it fits into these 
frameworks and the corresponding generic solutions can be directly applied. In the remaining part of this 
paper, we will derive several structural results for a particular type of multi-user interaction scenario. 

B. Additively Coupled Sum Constrained Games 

In this subsection, we present the definition of ACSCG and subsequently, we present several exemplary 
multi-user scenarios which appertain to this new class of game. 

Definition 1: A multi-user interaction T = (M,A,u) is a ACSCG if it satisfies the following assump- 
tions: 

Al: Vn G N, action set A n C 1Z is defined to bd_ 

A 



A n = [(ai, a 2 n ,--- , a*) \ a k n G [< fc n , a£f ] and ^ a k n < M n }. (3) 

k=i 

A2: There exist h n : K -)■ 11, /* : A- n -)■ 11, and g k n : A- n -+ 1Z, k = 1, . . . , K, such that 

K 
U n (a) = J2 [h k n{a k n + /n(a-n)) - <?n(a-n 



K 
fe=l 



for all a G A and n G J\f. h k L (-) is an increasing, twice differentiable, and strictly concave function and 
f k (-) and g k (-) are both twice differentiable. 
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f : R n -> R is convex if dom/ is a convex set and f(6x + (1 - 0)y) < 8f(x) + (1 - 0)f(y), Vx, y € dom/, < 6 < 1. 
5 We consider a sum constraint throughout the paper rather than a weighted-sum constraint, because a weighted-sum constraint 
can be easily converted to a sum constraint by rescaling An- Besides, we nontrivially assume that 2fc=i a n',fc X — M n - 



The ACSCG model defined by assumptions Al and A2 covers a broad class of multi-user interactions. 
Assumption Al indicates that each player's action set is a K-dimensional vector set and its action vector 
is sum-constrained. This represents the communication scenarios in which each user needs to determine 
its multi-dimensional action in various channels or networks while the total amount of resources it can 
consume is constrained. Assumption A2 implies that each user's utility is separable and can be represented 
by the summation of concave functions foj* minus "penalty" functions g* across the K dimensions. In 
particular, within each dimension, the input of h\ is an additive combination of user n's action o^ and 
function /^(a_ n ) that depends on the remaining users' joint action a_„. Since a^ only appears in the 
concave function /ijj, it implies that each user's utility is concave in its own action, i.e. diminishing 
returns per unit of user n's invested action a n , which is common for many application scenarios in 
communication networks. 

Summarizing, the key features of the game model defined by Al and A2 include: each user's action 
is subject to a sum constraint; users' utilities are impacted by additive combinations of o^ and /^(a_ n ) 
through concave functions h^. Therefore, we term the game V that satisfies assumptions Al and A2 
as ACSCG. In the following section, we present several illustrative multi-user interaction examples that 
belong to ACSCG. 

C. Examples of ACSCG 

We present four examples that satisfy assumptions Al and A2 and belong to ACSCG. The details of 
functions /i*(«), /„(•) and g^(-) in each example are summarized in Table U For each example, Table U 
also summarizes the applicable convergence conditions that will be provided in the remaining parts of 
the paper. 

Example 1: We first consider a simple two-user game with two-dimension action spaces, i.e. N = 
K = 2. The utility functions are given bjo 

u n (a) = -expj-4 - ^(al n ) 2 + 1 + ^(a 2 _J 2 + l} - expj-a 2 , + ^(alj 2 + 1 - ^(a 2 _J 2 + l}, 

for n = 1, 2. The resource constraints are Ylk=l a n ^ M n in which M n > and a^ > for Vn, k. 

Example 2: (Power control in frequency-selective Gaussian interference channel H17V ([20]) There are 
A^ transmitter and receiver pairs in the system. The entire frequency band is divided into K frequency 
bins. In frequency bin k, the channel gain from transmitter i to receiver j is denoted as Hf-, where k = 

6 In this example, since there are only two users, the subindex —n denotes the user but n. 



TABLE I 
EXAMPLES[3[4]AS ACSCG. 



Examples 


/5(a-„) 


/!*(») 


ffn(a-n) 


Convergence 
conditions 


Example [JJ 


^(a_„) = ^/(aL n ) 2 + l-^/(a 2 _J2 + l 
/ 2 (a_„) = y / (a 2 _ n )^ + l-^(a 1 _J2 + l 


-e"* 





d 


Example [2] 




Iog 2 (a£ + #*„*) 


l0g 2 (a*+ E ^m„P™) 


JcB-lfC8b 


Example [3] 


E %^i> k m 


i 





dCB-fC8t 


^rt "Wbb 1 


Example [4] 




log 2 (o£ +#*„«) 


l0g 2 (<7* +#*„/* (a_ n )) 


JC4t-dC8b 



1, 2, • • • , if. Similarly, denote the noise power spectral density (PSD) that receiver n experiences as a k and 
player n's transmit PSD as P*. The action of user n is to select its transmit power P„ = [P^ P% ■ ■ ■ P^] 
subject to its power constraint: Ylk=l ^n — P« ax - F° r a fixed P n , if treating its interference as noise, 
user n can achieve the following data rate: 

MP)=E'°ft(i+ Jfc+E g "fi tP o 

fc=l m=l m^n 

Example 3: (Delay minimization in Jackson Networks /|2?5|/) As an additional example, we consider 
a network of N nodes. A Poisson stream of external packets arrive at node n with rate ijj n and the 
input stream is split into K traffic classes, which are individually served by exponential servers. Denote 
node n's input rate and service rate for class k as ip* ar *d fJ-n respectively. Therefore, the action of 
node n is to determine the rates for different traffic classes \t n = [ip^ 4>n" ' Tpn] anc ' tne tota l rate 
is subject to the minimum rate constraint: J2k=i V'n ^ V'™ m - The packets of the same traffic class 
constitute a Jackson network in which Markovian routing is adopted: packets of class k completing 
service at node m are routed to node n with probability r k nn or exit the network with probability 



'mO 



1 — Yln=i r mn- Denote the arrival rate for class k at node n as rj k . By Jackson's Theorem, 



N 



we have rfc = #J + £ m=1 vi 



fc Jfe 



,n 



1,2,- •• ,K. Denote [R fe ], 



r k nm , T k = (I - R fe )-\ and 



^iV 



uj,„ = [T ] nm . Equivalently, we have r/ k = J2 m =l v mn4 ) m,- Each node aims to minimize its total M/M/l 



queueing delay incurred by accommodating its traffic: 

K 1 

d n (*) = y = . (6) 

k=l rn zL/m=l u mn K rm 

Example [3] can be shown to be a special case of ACSCG by slightly transforming the action sets and 
utilities. We can define user n's action as — ^ n . For user n, the sum constraint becomes Y^,k=i ~^n ^ 
—ipn m an d minimizing d n (^f) is equivalent to maximizing — d„(S&). 

Example 4: (Asynchronous transmission in digital subscriber lines network [19]) The basic setting 
of this example is similar to that of Example [2] except that inter-carrier interference (ICI) exist among 
different frequency bins. Due to the loss of the orthogonality, the interference that user n experiences in 
frequency bin k is 

K 

/ n fe (p_ n ) = J2 (E ?(* - j) H L n pi) , (7) 

m^n j=l 

in which j(j) is the ICI coefficient that represents the relative interference transmitted signal in a particular 
frequency bin generates to its jth neighbor bin. In particular, it takes the form 

1, if J = 

7(J)H 2 K K , (8) 



^sin^ij)' 2 -J - 2 

It satisfies the symmetric and circular properties, i.e. j{—j) = *y(j) = j(K — j). User n's achievable 
rate in the presence of ICI is given by 

K 

r„(P) = £>. 



fc=i 



TIK pfc 

-i _|_ rL nn r n 



< + E^jEpl 7(fc " J)^ni^ 



(9) 



D. Issues related to ACSCG 



Since the ACSCG model represents a good abstraction of numerous multi-user resource allocation 
problems, we aim to investigate the convergence properties of various distributed algorithms in ACSCG 
without and with real-time message passing. 

ACSCG is a concave game (6l Q and therefore, it admits at least one pure NE. In practice, we want 
to provide the sufficient conditions under which best response dynamics provably and globally converges 
to a pure NE. However, the existing literature, e.g. the diagonal strict concavity (DSC) conditions in 
iTTTl and the supermodular game theory (U- ifTOl , does not provide such convergence conditions for the 
general ACSCG model. For example, the DSC conditions developed for general concave games do not 
guarantee the convergence of best response dynamics [7]. Even if the utility functions in ACSCG possess 
the supermodular type structure, due to the sum constraint, the action set of each user is generally not a 
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sublatticqj of 72. . Therefore, the convergence results based on supermodular games cannot be directly 
applied in ACSCG. On the other hand, if we want to maximize the sum utility by enabling real-time 
message passing among users, we also note that, the utility u n is not necessarily jointly concave in a 
because of the existence of <7„(-). Therefore, the existing algorithms developed for the convex NUM are 
not immediately applicable either. 

In fact, a unique feature of the ACSCG is that different users' actions are additively coupled in 
/i^(-) and each user's action space is sum-constrained. In the following sections, we will fully explore 
these specific structures and address the convergence properties of various distributed algorithms in two 
different scenarios. Specifically, Section HIT] investigates the scenarios in which each user n can only 
observe {/n(a-n.)}|Li an ^ cannot exchange any information with any other user. Section ITVl focuses on 
the scenarios in which each user n is able to announce and receive information in real-time to and from 
the remaining users about ^ and u ^[ , Vm / n,k = 1, . . . ,K. 

III. Scenario I: no message exchange among users 

In communication scenarios where users cannot exchange messages to achieve coordination, the 
participating users can simply choose actions to selfishly maximize their individual utility functions 
u n (&) without taking into account the utility degradation caused to the other users. In particular, each 
user individually solves the following optimization program: 

max u„(a). (10) 

a„G.4„ 

The steady state outcome of such a multi-user interaction is usually characterized as a NE, at which 
given the other users' actions, no user can increase its utility alone by unilaterally changing its action. 
It is worth pointing out that, since there is no coordination signal among users, NE generally does not 
lead to a Pareto-optimal solution. Section IV will discuss distributed algorithms in which users exchange 
coordination signals in order to improve the system efficiency. 

A. Properties of Best Response Dynamics in ACSCG 

To better understand the key properties of the ACSCG, in this subsection, we first focus on the scenarios 
in which /^(a_ n ) is the linear combination of the remaining users' action in the same dimension k, i.e. 

/n(a-n) = Yl F ™na k m (H) 

7 In supermodular games, for each player, the action set is a nonempty and compact sublattice of TZ , We can verify that 
with the sum constraint, A n is usually not a sublattice of 1Z K by taking the component-wise maximum. 
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and F^ nn E 1Z, Vm, n, k. Specifically, both Example [2] and [3] in Table U belong to this category. In Section 
IIII-BI we will extend the results derived for the functions /*(a_ n ) defined in (fTT|) to general /*(a_ n ). 
Since h k t (-) is concave, the objective in (fTOl) is a concave function in a k n when the other users' actions 
a_„ are fixed. To find the globally optimal solution of the problem in ( fTOb . we can first form its Lagrangian 

K 

L n (a n , A) = u„(a) + A(M n - J] a*), (12) 

fc=i 

in which a„ G [ a «™> a nfc X ]- By taking the first derivatives of (fT2l) . we have 

<9L n (a n ,A) _ dh n (a n + Y Jm ^n F mn a m) 

T~£ ~~ a~ b A — U. Iyl5) 



Denote 

(14) 



^ (a -- A) -[{^}' 1(A) -^ F ^ a: 



a/i: 



in which {-gjr} is the inverse functional of -g£- and [x]^ = max{min{x,a}, b}. The optimal solution 
of (fTOl) is given by a* fc = Z^(a_ n , A*), where the Lagrange multiplier A* is chosen to satisfy the sum 
constraint Y^k=i a *n = M n- 

We define the best response operator B k (-) as 

B£(a_ n ) = Z*(a_ m A*). (15) 

We consider the dynamic adjustment process in which users revise their actions over time based on 
their observations about their opponents. A well-known candidate for such adjustment processes is the 
so-called best response dynamics. In the best response algorithm, each user updates its action using the 
best response strategy that maximizes its utility function in (0]). We consider two types of update orders, 
including sequential update and parallel update. Specifically, in sequential update, individual players 
iteratively optimize in a circular fashion with respect to their own actions while keeping the actions of 
their opponents fixed. Formally, at stage t, user n chooses its action according to 

a ri = -^n([ a l) • • • ) a n-l) a n.+ D • • • ) a AT ])• (16) 

On the other hand, players adopting the parallel update revise their actions at stage t according to 

a k / = BftaL- 1 ). (17) 

We obtain several sufficient conditions under which best response dynamics converges. Similar con- 
vergence conditions are proved in |[T8l - ||20l for Example |2] in which h^(x) = log 2 (a^ + H k n x). We 

8 If $ x = x* such that -q£-\ x =x* = -V we let {"af"} M — — oo. 
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TABLE II 
Comparison among conditions JC1I >-i 1C6I) , 



Conditions 


Assumptions about /^(a_„) 


ft* (a;) 


Measure of residual error a„ — a^ 


Contraction factor 


CD 


GD 


A2 


1-norm 


2p(T max ) 


o 


ifTTTl and F^ n have 
the same sign for Vfc, m 7^ n 


A2 


1-norm 


p(T max ) 


o 


O 


© 


weighted Euclidean norm 


p(S max ) 


d 


general 


A2 


1-norm 


2p(T max ) 


o 


— , , have the same sign 
for Va £ .4, fc, k' ,m ^ n 


A2 


1-norm 


p(f max ) 


cu 


general 


© 


weighted Euclidean norm 


P (s max ) 



consider more general functions h^(-) and further extend the convergence conditions in lfl"8l - EOll . The 
key differences among all the sufficient conditions which will be provided in this section are summarized 
in Table HI 

1) General h^(-): The first sufficient condition is developed for the general cases in which the functions 
hn(-) in the utilities u n (-) are specified in assumption A2. Define 



maxfc IF^J, ifm^n 
0, otherwise, 

and let / o(T max ) denote the spectral radius of the matrix T max . 
Theorem 1: If 

p(T m3X ) < 



(18) 



1 
2' 



(CI) 



then there exists a unique NE in game T and best response dynamics converges linearly to the NE, for 
any set of initial conditions belonging to A with either sequential or parallel updates. 

Proof: This theorem is proved by showing that the best response dynamics defined in (fl6l ) and (fTTT ) 
is a contraction mapping under (IC1I ). See Appendix A for details. ■ 

In multi-user communication applications, it is common to have games of strategic complements (or 
strategic substitutes), i.e. the marginal returns to any one component of the player's action rise with 
increases (or decreases) in the components of the competitors' actions ||27l . For instance, in Examples [2] 
and|U increasing user n's transmitted power creates stronger interference to the other users and decreases 
their marginal achievable rates. Similarly, in Example |3l increasing node n's input traffic rate congests 
all the servers in the network and increases the marginal queueing delay. Mathematically, if u n is twice 
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differentiable, strategic complementarities (or strategic substitutes) can be described as 

d 2 n n (an,a„ n ) ^ Vm ^ n ^ fc> (or ^^K, a_ n ) ^ ^ Vm _, n ^. fe) (19) 

dabdcim dalida^ 

We can verify that Examples [2J |3l and @] are games with strategic substitutes. For the ACSCG models 
that exhibit strategic complementarities (or strategic substitutes), the following theorem further relaxes 
condition (1C11 ). 

Theorem 2: Let Y be an ACSCG with strategic complementarities (or strategic substitutes), i.e. F^ n < 
0, Vfc, m^n, (or F^ n > 0, Vfe, m 7^ n). If 

p(T max ) < 1, (C2) 

then there exists a unique NE in game T and best response dynamics converges linearly to the NE, for 
any set of initial conditions belonging to A with either sequential or parallel updates. 
Proof: This theorem is proved by adapting the proof of Theorem [TJ See Appendix B. ■ 
Remark 1: (Implications of conditions dCil ) and rtC2D ) Theorem Q] and Theorem [2] give sufficient con- 
ditions for best response dynamics to globally converge to a unique fixed point. Specifically, maxj. | F^ n \ 
can be regarded as a measure of the strength of the mutual coupling between user m and n. The intuition 
behind (IC1I ) and (IC2b is that, the weaker the coupling among different users is, the more likely that best 
response dynamics converges. Consider the extreme case in which F^ n = 0,\/k,m ^ n. Since each 
user's best response is not impacted by the remaining users' action a_ n , the convergence is immediately 
achieved after a single best-response iteration. If no restriction is imposed on F^ n , Theorem Q] specifies a 
mutual coupling threshold under which best response dynamics provably converge. The proof of Theorem 
[T]can be intuitively interpreted as follows. We regard every best response update as the users' joint attempt 
to approach the NE. Due to the linear coupling structure in (TTTb . user n's best response in ([141) contains 
a term 2^2 m ^n ^mn a m tnat i s a li near combination of a_ n . As a result, the residual error |a^ +1 — a^L, 
which is the 1-norm distance between the updated action profile a^ +1 and the current action profile a f n , 
can be upper-bounded using linear combinations of |a^ — a^ 1 1 , in which m ^ n. Recall that F^ nn can be 
either positive or negative. We also note that, if a^ 7^ a^ 1 , a^ — a^ 1 contains both positive and negative 
terms due to the sum-constraint. In the worst case, the distance |a^ +1 — a^L is maximized if {F^n} 
and {cinl — a^i~ } are co-phase multiplied and additively summed, i.e. ^(a™ — am~ ) > 0, for 
\/k = 1, . . . , K, m 7^ n. After an iteration, all users except n contributes to user n's residual error at stage 
t + 1 up to >~2 m ^ n 2maxfc |-Pmn|| a m ~~ a m 1 |i- Under condition (1C1I) . it is guaranteed that the residual 
error contracts with respect to the special norm defined in (I67T ). Theorem [2] focuses on the situations in 
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which the signs of F mn are the same, Vm 7^ n, k. In this case, {F mn } and {0™ - a m ~~ } cannot be 
co-phase multiplied. Therefore, the region of convergence enlarges and hence, condition (1C2I) stated in 
Theorem |2] is weaker than condition (1C1I) in Theorem [Q 

Remark 2: (Relation to the results in references / [7ffl7 - /(201/) Similar to |[T8ll |[T9|| , our proofs choose 1- 
norm as the distance measure for the residual errors a^ +1 — a n after each best-response iteration. However, 
by manipulating the inequalities in a different way, condition (IC2b is more general than the results in [18] 
||T9l , where they require max^ F mn < -atzt- Interestingly, condition (lC2b recovers the result obtained in 
||20l where it is proved by choosing the Euclidean norm as the distance measure for the residual errors 
a^ +1 — a£j after each best-response iteration. However, the approach in |[20l using the Euclidean norm 
only applies to the scenarios in which /ijj(-) is a logarithmic function. We prove that condition (lC2b 
applies to any h^(-) that is increasing and strictly concave. 

2) A special class of h^(-): In addition to conditions (IClb and (1C2K we also develop a sufficient 

convergence condition for a family of utility functions parameterized by a negative number 9. In particular, 

h*(-) satisfied 

, I logfaj; + F*Lx), if 9 = -1, 

h k n(x) = < a "*„;+/ (20) 



(K+ F L x ) 



if -1< 9 < or 9 < -1. 



3+1 

and a' £?l and F^ n > 0. The interpretation of this type of utilities has been addressed in II28T1 . It is shown 
that varying the parameter 9 leads to different types of fairness across a^ + F^ n (a^ + J2 m ^n ^mn a m) 
for all k. In particular, 9 = — 1 corresponds to the proportional fairness; if 9 = — 2, then harmonic mean 
fairness; and if 9 = —00, then max-min fairness. We can see that, Examples [2] and [3] are special cases 
of this type of utility functions. In these cases, best response dynamics in equation (IT4T > is reduced to 



'nl 3 — ni A) 



Define 



j max] 
' \mn 



\) = 


vpfe J A pk 

-*■ nn M nn 


_Vf* a k 




T 1 k = i( F "L m ) 1+1! [\pk 


[f£J }' lim ^ n 




0. 




otherwise 



(21) 



(22) 



For the class of utility functions in d20l ). Theorem [3] gives a sufficient condition that guarantees the 
convergence of the best response dynamics defined in (|2T1 ). 

9 If a* + F, n n x < 0, we let h^(x) — — 00. We assume for this class of /l„(-) that for Va_„ 6 .A-n, there exists a n 6 ^4 n 
such that a£ + F,j„x > for Vn, fc. 
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Theorem 3: For h^(-) defined in (l20l ). if 

p(S max ) < 1, (C3) 

then there exists a unique NE in game T and best response dynamics converges linearly to the NE, for 
any set of initial conditions belonging to A and with either sequential or parallel updates. 

Proof: It can be proved by showing that the best response dynamics defined in (l2"Tb is a contraction 
mapping with respect to the weighted Euclidean norm. See Appendix C for details. ■ 

Remark 3: (Relation between conditions ( ICJI ) and the results in reference [20]) For aforementioned 
Example [2j Scutari et al. established in EUI a sufficient condition under which the iterative water- 
filling algorithm converges. The iterative water-filling algorithm essentially belongs to best response 
dynamics. Specifically, in |[20l . Shannon's formula leads to 6 = — 1 and cross channel coefficients satisfy 
F mn ^ 0) ^k> m ¥^ n - Equation (|2TT) reduces to the water-filling formula 

(23) 



l n ( a -n, A) — — — -— £ 2_, ^mi 



and [S max ] mn = maxfcF^ n . By choosing the weighted Euclidean norm as the distance measure for the 
residual errors a^ +1 — a l n after each best-response iteration, Theorem [3] generalizes the results in GUI for 
the family of utility functions defined in (l20l ). 

Remark 4: (Relation between conditions riCil ). ( IC2I) a«J rtCJI )) The connections and differences between 
conditions (IClb . (IC2b and (IC3I ) are summarized in Table ITT] We have addressed the implications of (IClb 
and (IC2I ) in RemarkQ] Now we discuss their relation with (IC3I ). First of all, condition (IClb is proposed for 
general /&*(•) and condition (IC3b is proposed for the class of utility functions defined in (l20l ). However, 
Theorem [T]and Theorem [3] individually establish the fact that best response dynamics is a contraction map 
by selecting different vector and matrix norms. Therefore, in general, (IClb and (IC3b do not immediately 
imply each other. Note that [S max ] mn < Q mn ■ max^ \F mn \ in which Q mn satifies 



i+l 



mm J 



» n " v^it /pfc si+i fc (F k U+| L • (Fit lF k ^ 1+i 

Z^fc=l\ nn) \ mm) ^-^kxnnlmm) 



(24) 



The physical interpretation of £ mn is the similarity between the preferences of user m and n across the 
total K dimensions of their action spaces. Recall that both S max and T max are non-negative matrices and 
gmax j s e i ernen t- W ise less than or equal to max m -^ n £ mn T max . By the property of non-negative matrix 
and condition (IClb . we can conclude /)(S max ) < /)(max m ^ n CmnT max ) < max m ^ n &f-. The relation 
between (IC lb and (IC3b is pictorially illustrated in Fig. [Q Specifically, if users have similar preference in 
their available actions and the upper bound of £ mn that measures the difference of their preferences is 
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In general 




)!+i 



max fc,m^Ti [fnn I * 'mm . 
nilll^ ^rn^n\" nn I ram,) ° 



<2 



Fig. 1. Relation between dClt and jC3t . 



below the following threshold: 



max fc,m^n(,-f'nn/-^mmj ' 9 



(25) 



we know that dCTT ) implies dC3b in this situation because p(S max ) < max m>n ( mn ■ p(T max ) < 2 ■ | = 1. 
We also would like to point out that, the LHS of (|25T ) is a function of and the LHS = 1 if 9 = — 1. 
When = — 1, T max coincides with S max . Mathematically, in this case, (IC3b is actually more general 
than (1C2I) . because it still holds even if coefficients F^ nn have different signs. 



B. Extensions to General /„(•) 

As a matter of fact, the results above can be extended to the more general situations in which /%(■) is 
a nonlinear differentiable function, Vn, k and its input a_ n consists of the remaining users' action from 
all the dimensions. Accordingly, equation ([141 becomes 

The conclusions in Theorem [Q |2l and [3] can be further extended as Theorem 01 and [5J [6] that are listed 
below. We only provide the proof of Theorem [4] in Appendix D. The detailed proofs of Theorem [5] and 
[6] are omitted because they can be proven similarly as Theorem [4] 



{§}' l m -/„<(*-„) 



(26) 
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For general /*(•), we denote 



max aGAfc , ^f =1 



Besides, for h 



denned in (1201) . we define 

Ef =1 (^ m ) 1+ ^ 



Ef-lWJn) 1 



t- max ae ^ jfe , 



0, 






0. 



a/J(«-») 

9 a m 


if ?n 7^ re 




otherwise. 




L 9»m 


teH- 


if m^n 
otherwise. 



(27) 



Theorem 4: If 



P(T D 



1 
<2' 



(28) 



(C4) 



then there exists a unique NE in game T and best response dynamics converges linearly to the NE, for 
any set of initial conditions belonging to A with either sequential or parallel updates. 

Proof: This theorem can be proved by combining the proof of Theorem Q] and the mean value theorem 
for vector-valued functions. See Appendix D for details. ■ 

Similarly as in Theorem [H for the general ACSCG models that exhibit strategic complementarities (or 
strategic substitutes), we can further relax condition dC4b . 

Theorem 5: For T with strategic complementarities (or strategic substitutes), i.e. J "\7 > 0,Vm f= 



n, 



,k,k',aeA, (or 8/ ^ (a fc 7" ) < 0,Vm ^ n, k,k',a € A), if 



da 



P {T 



<1, 



(C5) 



then there exists a unique NE in game T and best response dynamics converges linearly to the NE, for 
any set of initial conditions belonging to A with either sequential or parallel updates. 
Theorem 6: For h^(-) defined in (1201 ). if 



p(S r 



<1, 



(C6) 



then there exists a unique NE in game V and best response dynamics converges linearly to the NE, for 
any set of initial conditions belonging to A with either sequential or parallel updates. 
Remark 5: (Implications of conditions f \C4\) , f |C5| ). and f |C6D ) Based on the mean value theorem, we 



know that the upper bound of the additive sum of first derivatives ^ 



K 
fe=l 



d/„ fc (a- n ) 
da**" 



governs the maximum 



impact that user m's action can make over user re's utility. As a result, Theorem SI Theorem [51 and 



Theorem [6] indicate that ^ 



K 

fc=l 



d/*(a-n) 
da* 



can be used to develop similar sufficient conditions for the 



global convergence of best response dynamics. Table [TT] summarizes the connections and differences 
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among all the aforementioned conditions from (IC1I) to (IC61) . We can verify that, for the linear function 
fn{) that is defined in (flT]) and studied in Section UlI-AI Va G A, m ^ n, it satisfies 

5/ n fc (a_ n ) f F* n , if k' = k 



' a ™ " 0, otherwise. 

In addition, we can see that, in Example [4] f^(-) is actually an affine function with 

df^V-n) = i y(k - k')H k mn , if k' = k 
""m [ 0, otherwise, 

and S max is reduced to 



(29) 



(30) 



rgnii-x! A I maX fc' E£=l ^(^ - feO-^mn. if m ¥= n „ n 

P Jmra — S l-' 1 ) 

0, otherwise. 

As an immediate result of Theorem [6l we have the following corollary which specifies a sufficient condi- 
tion that guarantees the convergence of the iterative water-filling algorithm for asynchronous transmissions 
in multi-carrier systems lfT9l . 

Corollary 1: In Example [4] if the matrix S max defined in (|3~TT) satisfies 

p(S max ) < 1, (32) 

then there exists a unique NE in game T and the iterative water-filling algorithm converges linearly to 
the NE, for any set of initial conditions belonging to A and with either sequential or parallel updates. 

Remark 6: (Impact of sum constraints) An interesting phenomenon that can be observed from the 
analysis above is that, the convergence condition may depend on the maximum constraints {M„}^ =1 . 
This differs from the observation in J20l that the presence of the transmit power and spectral mask 
constraints does not affect the convergence capability of the iterative water-filling algorithm. This is 
because when functions /^(a_ n ) are affine, e.g. in Example [2j [3l and |U the elements in T max an d 
S max are independent of the values of {M n }„ =1 . Therefore, dCll )- dC6l ) are independent of M n for affine 
/^(a_ n ). However, for non-linear /^(a_ n ), the values of {M n }^ =1 specify the range of users' joint 
feasible action set A, and this will affect T max and S " accordingly. In other words, in the presence 
of non-linearly coupled /*;(a_ n ), convergence may depend on the players' maximum sum constraints 



C. Connections to the Results of Rosen [7] and Gabay / [2?l/ 

In Q, Rosen proposed a continuous-time gradient projection based iterative algorithm to obtain a pure 
NE under the assumption of DSC conditions. Here we present a discrete version of the algorithm in Q, 
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named "gradient play". Specifically, at stage t, each user first determines the gradient of its own utility 
function u n (a n ,aLLn)- Then each user updates its action a*j using gradient projection according to 



ou n (a. n: a 



& n ' — & n ' -r K n — -r w-^J 



and 



a n' a n' ' ' ' a n 



A , (34) 

An 



a n — [ a n a n " ' a n\ 

where n n is the stepsize and [v]^ denotes the projection of the vector v onto user n's action set A n with 
respect to the Euclidean norm || • H2. If K n is chosen to be sufficiently small, gradient play approximates 
the continuous-time gradient projection algorithm. For each nonnegative vector re = [k\ . . . kjv], define 

3(a,re) = [KiViMi(a) K 2 V 2 «2(a) ... k n V N u N (a)f '. (35) 

The definition of DSC in Q is that, for fixed re > and every a , a 1 G A, we have 

(a 1 - a°) T 5 (a°,re) + (a - a 1 ) T 5 (a 1 ,re) > 0. (36) 

A sufficient condition for DSC is that the symmetric matrix G(a, re) + G T (a, re) be negative definite for 
a£i, where G(a, re) is the Jacobian with respect to a of ^(a, re). 

However, when using gradient play to search for a pure NE, the stepsize K n needs to be carefully chosen 
and set to be sufficiently small, which usually slows down the rate of convergence. As an alternative 
distributed algorithm, for concave games with A n = 1Z + , Vn G J\f, Gabay and Moulin provided in E4l 
a dominance solvability condition under which best response dynamics globally converges to a unique 
NE. Specifically, the dominance solvability condition is given by 



d 2 u n y—^ I d 2 u. 



> 



E 



d 2 a n *—f I da n da 



in 



(37) 



The sufficient conditions provided in this section and Gabay 's dominance solvability condition specify the 
convergence conditions of best response dynamics in different subclasses of concave games. Specifically, 
our results are developed for concave games in which every user has a multi-dimensional action space 
subject to a single sum-constraint and Gabay's dominance solvability condition is proposed for concave 
games with single dimensional strategy. 

D. Connections to Linearly Coupled Communication Games 

We investigated in [29] the convergence properties in certain communication scenarios, namely linearly 
coupled communication games (LCCG), in which each user has a convex action set A n C TZ+ and the 
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TABLE III 

A SUMMARY OF VARIOUS CONVERGENCE CONDITIONS IN CONCAVE GAMES. 



Algorithms 


Sufficient conditions and the applicable games 


Gradient play 


Rosen's DSC conditions for concave games (7) 


Best response 


Gabay's dominance solvability condition for concave games 
with A n = TZ+ (24), conditions OhOU for ACSCG 



utility functions take the form 



N 



i (a) = o^" • (/x - y] T m a m ). 



m=l 



(38) 



It has been used to model the flow control mechanism in communication networks [30] . In best response 
dynamics, at stage t, user n chooses its action according to 



iW -1 ) 



/3n(M"E 



m£j\f\{n} 'm u m 



t-l> 



(39) 



T„(l+/3 n ) 

We can see that, LCCG is similar to ACSCG in the sense that the best response iterations at stage t 
in (fl4l and d39l both contain the linear combinations of a* -1 . However, since A n C TZ in LCCG, we 
can explicitly derive the Jacobian matrix for best response dynamics and determine the exact locations 
of all its eigenvalues. Consequently, we are able to develop the necessary and sufficient condition that 
ensures the spectral radius of the Jacobian matrix to be less than 1 and best response dynamics globally 
converges. However, in ACSCG, due to the sum-constraint, there exists a non-linear operation [x]% in 
equation ([I4l . This complicates the analysis of the Jacobian matrix's eigenvalues. Therefore, we usually 
choose various appropriate matrix norms to bound the spectral radius of the Jacobian matrix and ensure 
the best response iteration to converge under these matrix norms. This approach generally results in 
various sufficient, but not necessary, conditions. 



IV. Scenario II: message exchange among users 

In this section, our objective is to coordinate the users' actions in ACSCG to maximize the overall 
performance of the system, measured in terms of their total utilities, in a distributed fashion. Specifically, 
the optimization problem we want to solve is 



A? 



max 

aeA 



J^u n (a). 



(40) 



71=1 



We will study two distributed algorithms in which the participating users exchange price signals that 
indicate the "cost" or "benefit" that its action causes to the other users. Allocating network resources via 
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pricing has been well-investigated for convex NUM problems lfl5Tl . where the original NUM problem 
can be decomposed into distributedly solvable subproblems by setting price for each constraint resource, 
and each subproblem has to decide the amount of resources to be used depending on the charged price. 
However, unlike in the conventional convex NUM, pricing mechanisms may not be immediately applicable 
in ACSCG if the objective in d40b is not jointly concave in a. Therefore, we are interested in characterizing 
the convergence condition of different pricing algorithms in ACSCG. 

We know that for any local maximum a* of problem (l40l ). there exist Lagrange multipliers X n , v^, ■ ■ • , v^ 
and v n , • • • , v'n such that the following Karush-Kuhn-Tucker (KKT) conditions hold for all n E M: 

K 

A„($>**-M n ) =0, A n >0 (42) 

fe=l 

*** - <D = 0, v n k «£ - a k n *) = 0, v k n1 v' k > 0. (43) 

Denote 7r^ n user m's marginal fluctuation in utility per unit decrease in user n's action a^ within the 
kth dimension 

■K k (a k a k ) = dUm ^ (44) 

"mn\ u 'mi a —mJ a j~ ' V^^^ 

which is announced by user m to user n and can be viewed as the cost charged (or compensation paid) 
to user n for changing user m's utility. Using (1441 . equation (|4TT > can be rewritten as 

du n {a*) 






^2 ^mn^m^-m) = A n + f^ - V k . (45) 



If we assume fixed prices {7r^ n } and action profile a^ n , condition (I43T ) gives the necessary and sufficient 
KKT condition of the following problem: 



K 

: Mr, (a) — 

a„G.A 



1 ' i;1 n u n (a) - Y^ «n • ( Yl n ™) ■ (46) 

fc=l mj^n 

At an optimum, a user behaves as if it maximizes the differences between its utility minus its payment to 
the other users in the network due to its impact over the other users' utilities. Different distributed pricing 
mechanisms can be developed based on the individual objective function in (l46l) and the convergence 
conditions may also vary based on the specific action update equation. 

When optimization program (l40l) is not convex, the pricing algorithms developed for convex NUM, 
e.g. gradient and subgradient algorithms, cannot be directly applied. In the next two subsections, we 
will investigate two distributed pricing mechanisms for non-convex ACSCG and provide two sufficient 
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conditions that guarantee their convergence. Specifically, under these sufficient conditions, both algorithms 
guarantee that the total utility is monotonically increasing until it converges to a feasible operating point 
that satisfies the KKT conditions. Similarly as in Section ITlI-AI we first assume /^(a_ n ) takes the form 
in (TTTb and users update their actions in parallel. 



A. Gradient Play 

The first distributed pricing algorithm that we consider is gradient play. The update iterations of gradient 
play need to be properly redefined in presence of real-time information exchange. Specifically, at stage 
t, users adopting this algorithm exchange price signals {^mn } using the gradient information at stage 
£ — 1. Within each iteration, each user first determines the gradient of the objective in (l46l ) based on the 
price vectors {nmn } and its own utility function ti n (a n ,a^ n 1 ). Then each user updates its action a^ 



using gradient projection algorithm according to 

/ ou n {3. n , a_ n , 



,'k,t 



a 



k,t-l 



+ K 



{- 



9 a n 



/ j ran 



and 



a n — Y a n a n ' ' ' a n\ 



LLrn ^*>>r) ' L4<n 



A„ 



(47) 



(48) 



in which the stepsize k > 0. The following theorem provides a sufficient condition under which gradient 
play will converge monotonically provided that we choose small enough constant stepsize k. 
Theorem 7: If Vn, k, x, y G A- n , 



d 2 h k (x) 
inf — " > -oo, and 
x d z x 



(C7) 



gradient play converges for a small enough stepsize k. 

Proof: This theorem can be proved by showing the gradient of the objective function in (|40T > is Lipschitz 
continuous and applying Proposition 3.4 in [31]. See Appendix E for details. ■ 

Remark 7: (Application of condition dC7D ) A sufficient condition that guarantees the convergence 
of distributed gradient projection algorithm is the Lipschitz continuity of the gradient of the objective 
function in (l40l) . For example, in the power control problem in multi-channel networks 1211 . we have 
h k n {x) = \og 2 {a k n + H k n x) and #£(P- n ) = log 2 0£ + Y, m ^n H mn P n\)- For this configuration, we can 
immediately verify that condition dC7b is satisfied. Therefore, gradient play can be applied. Moreover, as 
in EQ, if we can further ensure that the problem in (l40l) is convex for some particular utility functions, 
gradient play converges to the unique optimal solution of d40l ) at which achieving KKT conditions implies 
global optimality. 
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B. Jacobi Update 

We consider another alternative strategy update mechanism called Jacobi update [32 ]. In Jacobi update, 
every user adjusts its action gradually towards the best response strategy. Specifically, the maximizer of 
problem (l46l) takes the following form 

B'n k (*-n) = {^Y\^ +«*- # + E <n) " £ *£/&, (49) 

in which A n , v k , and z^ fc are the Lagrange multipliers that satisfy complementary slackness in ((42)) and 
(l43l) . and 7r^ n is defined in (l44l) . In Jacobi update, at stage £, user n chooses its action according to 

a k / = a*/- 1 + *[]%(&) - a k /~ l ] , (50) 

in which the stepsize k £ (0,1]. The following theorem establishes a sufficient convergence condition 
for Jacobi update. 

Theorem 8: If Vn, k,x,y 6 A- 



-ni 



mf — " > -oo, sup — " < 0, and 

x d A X x O z X 



V5n( x )~ V5n(y 



<L'||x-y||, (C8) 



Jacobi update converges if the stepsize k is sufficiently small. 

Proof: This can be proved using the descent lemma and the mean value theorem. The details of the 
proof are provided in Appendix F. ■ 

Remark 8: (Relation between condition ( IC8D and the result in /E2/j Shi et al. considered the power 
allocation for multi-carrier wireless networks with non-separable utilities. Specifically, u n (-) takes the 
form 

un(P) = n(j2lo g2 (l+ J?™ P * )V (51) 

y fc—i n t Z—tm^n 11 mn 1 ml 

in which n(-) is an increasing and strictly concave function. Since the utilities are non-separable, the 
distributed pricing algorithm proposed in [22], which in fact belongs to Jacobi update, requires only 
one user to update its action profile at each stage while keeping the remaining users' action fixed. The 
condition in (IC8b gives the convergence condition of the same algorithm in ACSCG. We prove in Theorem 
[7] that, if the utilities are separable, convergence can still be achieved even if these users update their 
actions at the same time. Therefore, we do not need an arbitrator to select the single user that updates 
its action at each stage. 

Remark 9: (Complexity of signaling) The complexity of message exchange measured in terms of the 
number of price signals to update in (l44l is generally of the order of 0(KN 2 ). It is worth mentioning 
that the amount of signaling can be further reduced to 0(KN) in the scenarios where g k (-) are functions 
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°f Em^n Frnn a m- ^ n ^ s case > eac h user on ly needs to announce one price signal -k^ for each dimension 

of its action space: 

^k/k a k \ du n (a.) 

7TnK> a -n) = "77^ TT k FT (52) 

\ 2—im^n mn a m) 

Consequently, 7r^ n can be determined based on 7r^j n = F^ m 7r^, which greatly reduces the overhead of 
signaling requirement. It is straightforward to check that only 0{KN) messages need to be generated 
and exchanged per iteration in both utility functions (f5]) and ©. 

Remark 10: (Extension to general cases) As a matter of fact, conditions (IC7b and (IC8b apply to a 
broader class of multi-user interaction scenarios, including the general model defined in ©. Specifically, 
as addressed in Remark|7J the Lipschitz continuity of the gradient of Yl n =l u n( a ) is sufficient to guarantee 
that gradient play with a small enough stepsize achieves an operating point at which KKT conditions are 
satisfied. In addition, we can use the same technique in Appendix F to show the convergence of Jacobi 
update given that sup^ — «?g < 0, Vn, k, and the gradient of J2 n =i u n( a ) * s Lipschitz continuous. 

V. Numerical Examples 

In Section ITl-Cl we present several illustrative examples of ACSCG. This section uses Examples [Q and 
[3] to illustrate the various distributed algorithms discussed in the paper. 

We start with Example[T]to verify the proposed convergence conditions of best response dynamics. Even 
though it is a simple two-user game with A n C 1Z 2 , existing results in the literature cannot immediately 
determine whether or not the best response dynamics in this simple game can globally converge to a NE. 
Specifically, in Example [Q we have 

dfn(a-n) ai n <9/^(a_„) a 2 _ n 



da l _ n [73. \2 , 1 ' da 2 _ 



'(aija + l aa -n ^(a 2 _ n ) 2 + l 

dfn( a -n) al n dfl (a_n) a 2 _ n 



(53) 



da -n ^/(al n ) 2 + 1 da ~n ^{a\) 2 + 1 

According the definition of (|27T ). we have 

max | max ae „4 }^ k=1 J2 ^ a i , max a6 _4 }_^ k 

2a\ 2ai 1 2M, 

VK)^+T aiG ^ V(^+iJ \/m?7T 



rrpHiaX 



J 12 

max < maxajg^j 



a/a* (»-..) 



(54) 



Similarly, we can obtain [T max loi = , 2 . Therefore, p(T max ) = , / , AM ^ M ^ it is easy to show 

that p(T max ) < 1 «■ (M? -j)(Af? -§) < |. By condition CI, we know that if (M x -|)(Af? -g) < f , 
the best response dynamics is guaranteed to converge to a unique NE. We numerically simulate a scenario 
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Fig. 2. Actions versus iterations in Example Q] 



with parameters M\ = | and M2 = 1 in which condition (IC4b holds. We generate multiple initial action 
profiles of a^ and a[], iterate the best response dynamics, and obtain the action sequences a* and a 2 - 
Fig. [2] shows the trajectories of a{ and a 2 ' for different realizations. We can see that, best response 
dynamics converges to a unique NE. If we set M\ = 2 and M2 = 1, condition (IC4b does not hold 
any more. We observe from simulations that in many circumstances the best response dynamics will not 
converge, which agrees with our analysis in Remark [6] 

Now we consider Example which is the problem of minimizing queueing delays in a Jackson 
network. In particular, we consider a network with N = 5 nodes and K = 3 traffic classes. The total 
routing probability 1 — r^ that node m will route packets of class k completing service to other nodes 
is the same for Vm e N. We varied the total routing probability 1 — r^ and generated multiple sets 
of network parameters in which r^ nn are uniformly distributed for n = 1,2,- •• , N, \i\ are uniformly 
selected in [4, 5] for Vn, k, and ip™ m are uniformly chosen in [0.6, 1] for n = 1, 2, • • ■ , N. 

First of all, we compare the range of validity of the proposed convergence conditions. As we mentioned 
before, we have F^ n = f~_^X-i\ m in this example. Note that (I - R fc )- 1 = I + E J =i( R ' c ) i and Rk 
is a non-negative matrix. Therefore, we can conclude F^ nn > 0,Vm ^ n,k. Moreover, since h^(x) = 
— JTTT — , we choose to compare conditions (IC2I ) and (1C31 ). In Fig. [3 we plot the probability that 
conditions (|C2| ) and dC3l ) are satisfied versus the total routing probability 1 — r^ . From Fig. |3j we 
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0.5 
1-r k „ 



(C2) 
(C3) 



Fig. 3. Probability of {C2]l and O} versus 1 - r^ for Vm, k, N = 5, K = 3. 



can see that the probability of guaranteeing convergence decreases as the routing probability 1 — r^ 
increases and condition (IC3b shows a similar but slightly broader validity than (IC2b . Fig.|4]shows the delay 
trajectories of three nodes using both sequential and parallel updates in a certain network realization in 
which (IC2b and (1C31 ) are satisfied. We can see that, the parallel update converges faster than the sequential 
update. 

In Fig. [3l we also note that the probability that (IC2I ) or (IC3b is satisfied transits very quickly from the 



almost certain convergence to the non-convergence guarantee as 1 



'mO 



varies from 0.5 to 0.58. Similar 



observations have been drawn in the multi-channel power control problem [20], where 6 = — 1 in (|2Qb and 
the probability that condition (IC3b is satisfied exhibits a neat threshold behavior as the ratio between the 
source-interferer distance and the source-destination distance varies. In Jackson networks, this threshold 
can be roughly estimated. Define [S fc ] mn = F^ n for m ^ n and [S fc ] nn = for n G M. If we fix 1 — r^ 

for Vm, k, we prove in Appendix G that p(S ) < -4 1 for \/k. Therefore, p(S k ) < 1 when r^ > 0.5. 

We would like to estimate p{T max ) and /)(S max ) based on p(S k ). Note that T max defined in dH is the 
element-wise maximum over S k for k = 1, 2, . . . , K. Since T max and S k are all non-negative matrices, 
we know that p(T max ) > max^ p(S k ). In addition, recall the effect of max mn Q mn discussed in Remark 
SI We can approximate p(S max ) defined in (|22b using p(S max ) w max mi „ ( mn max^ p(S ). Therefore, 
we expect that p(T max ) and p(S max ) exceeds 1 for r k n0 < 0.5, which agrees with our observation from 
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Sequential update 
Parallel update 



15 20 25 30 35 40 45 50 



Fig. 4. Delays of nodes versus iterations. 



Fig. [3] The physical interpretation is that, if the packets exit the network with a probability less than 
50% after completing its service, i.e. more than half of the served packets will be routed to other nodes, 
the strength of the mutual coupling among users becomes too strong and the multi-user interaction in 
Jackson networks will gradually lose its convergence guarantee. 

In addition, we numerically compare two distributed algorithms in which users pass coordination 
messages in real time, including Jacobi update and gradient play. Fig. [5] shows the delay evolution of 
both distributed solutions for a particular simulated network in which we set k = 0.2. We initialize 
the system parameters such that inf n ± A*n ~~ S m =i v mn l l ) m > anc ^ both conditions dC7b and dC8l ) are 
satisfied. We can verify that for Example [3l problem (|401 is in fact a convex program. Therefore, there 
exists a unique operating point at which KKT conditions (l4TT)-(|43l) are satisfied. We can see that, both 
algorithms cause the total delay to monotonically decrease until it reaches the same performance limit 
that is strictly better than NE. Using the same stepsize k, Jacobi update converges more quickly than 
gradient play in this example. Similar observations are drawn in the other simulated examples. This is 
because the update directions of these two algorithms are different. Jacobi update moves directly towards 
the optimal solution of (l46l ). which is a local approximation of the original optimization program in (l40l . 
whereas the gradient play algorithm simply updates the actions along the gradient direction of ( |40l) . 
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Gradient play 
Jacobi update 



100 150 

iteration 



Fig. 5. Illustration of convergence for gradient play and Jacobi update. 



VI. Conclusion 



In this paper, we propose and investigate a new game model, which we refer to as additively coupled 
sum constrained games, in which each player is subject to a sum constraint and its utility is additively 
impacted by the remaining users' actions. The convergence properties of various generic distributed 
adjustment algorithms, including best response, gradient play, and Jacobi update, have been investigated. 
The sufficient conditions obtained in this paper generalize the existing results developed in the multi- 
channel power control problem and can be extended to other applications that belong to ACSCG. 



Appendix A 
Proof of Theorem [I] 

The following lemma is needed to prove Theorem [TJ 

Lemma 1: Consider any non-decreasing function p(x) and non-increasing function q{x). If there exists 
a unique x* such that p(x*) = q(x*), and the functions p(x) and q(x) are strictly increasing and strictly 
decreasing at x = x* respectively, then x* = &rgm.in x {max{p(x),q(x)}}. 

Proof of Lemma [7} See Lemma 1 in [19]. ■ 

Denote a n ' as the action of user n in the kth dimension after iteration t. Recall that [/i^] (•) > 0, 
for Vn, k. Therefore, Ylk=\ ° n = M n is satisfied at the end of any iteration t for any user n. Define 
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[x] + = max{x,0} and [x] = max{— x,0}. It is straightforward to see that 



A 



A 



J>^ - a k /} + = £[«£* - a k /]-,Vn,t,t'. 



fe=l 



fe=l 



We also define 



and 



K 



fe=i 

A 



g"' i (x)^^[^(a*_ n ,x)-a^ 



(55) 



(56) 



(57) 



fe=i 



in which £*(•) is defined in (fl4l) . Since /i^(-) is a continuous increasing and strictly concave function, it 
is clear that {-g^} (•) is a continuous decreasing function. If p n,t (\ t r f 1 ) / (i.e. it has not converged), 
p n,t (x) (q n,t (x), respectively) is non-decreasing (non-increasing) in x, and strictly increasing (strictly 
decreasing) at x = A^ +1 . From (I55T ) it is always true that p n ' t {X t r ^ 1 ) = 9 n,4 (A^ +1 ). We first prove the 
convergence of the parallel update case in (fTTT ). For Vn, we have 



A 



£[« t+1 - <tf\- 
fc=l 

= max{ f>W - a^]+, ^k' t+1 " «#T} 
fe=l fc=i 

= m ax{p n ' t (Aj+ 1 ), ? n ' t (Aj+ 1 )} 

<max{^(Al),^(A^} 

A 



<max{^[E^ n (a^ - a^ 1 )] , £ [ £ F, 

fc=l mj^n fc=l m^n 

K K 

< max 4 2_^ 2_*/ rnn\ a rn ~ a m )\ ' /;/,, \^mn\ a n 

fc=l m^n k=l m^n 

K K 

1 / ,, / j \^mn\ a rri a rn )\ i / ,, / / -'mnv'^rr 



fC / *-• *^ 1 ^ rt /i,I -1- \ 



max- 



,fc,t-i> 



,fc,t-i> 



rnj^n k=l 



m^n k=l 
K 



K K 

< J2 m fc ax l^mnl • { J2 [ a ™ ~ a ™ ~'] + £ [ 



fe,i „k,t-l 



m^n 



k=l 
K 



k=l 



\a k,t — n k,t 1 



(58) 

(59) 
(60) 

(61) 
(62) 
(63) 
(64) 
(65) 



= J2 2m ^\ F mn\-J2[ C 

m^n k=l 

where (I58T ) and (l65l) follows from (|55T ). (|59l follows from the definition of p n '* and g n,t in (l56l ) and 
(I57T ). (l60l is due to Lemma 1 in which x = A^, (loTb follows from the definition of p n,t and q n,t , the 
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expression of a n ' in (fT71) . and the fact that [[x] b a — [y]^] < [x — y} + and [[x]„ — [y]„] < [s — y]~, 
(l62l is due to the fact that [x + y] + < [x] + + [y} + and [s + y]~ < [x]~ + [y]~, (l64l) follows by using 
Efc^yjfc] 4 " < J2k\ x k\\Vk\ = ^Ek\ x k\([yk] + + [Vk]~) < max fc |x fc |]ir fc ([y fc ] + + [Vk]~)- For user n, we 
define that e l n = dm — dm~ ■ Inequality (l65l) can be written as e^ +1 < Z]m^n[T max ]mn e m i n which 
T max is defined in Q]Q- 

Since T max is a nonnegative matrix, by the Perron-Frobenius Theorem |[3ll . there exists a positive 
vector v/ = [w\ . . . tov] such that 

ll T lloo.mat = M T ), (66) 

where || • ||*, mat is the weighted maximum matrix norm defined as 

IV 



iah w 



max — VrAlwtD,-, AeK M (67) 

i=l,2,.»,JV TO ^— ' 

i=i 

Define the vectors e' +1 = [e* +1 , e£ + , . . . , e^" 1 ] 7 " and e* = [e\, e^, . . . , e t N ] T . The set of inequalities in 
(l65l) can be expressed in the vector form as < e' +1 < T max e*. By choosing the vector w that satisfies 
||T max ||^, mat = p(T max ) and applying the infinity norm || • ||^, we obtain the following 

||e || 0O <2||T e|| 00 <2||T ||oo^natll e lloo> ( 68 ) 

Finally, based on (1651 ) and (|68T ). it follows that 

,t+l _t „t 



max^ = ||e i+1 ||* < 2||T max ||^ mat ||e i ||* < 2||T max ||^ mat • max^ = 2p(T max ) • max^ {W) 

neN W n ' n£j\f W n n£j\f W n 

Therefore, if ||T max ||^ mat = p(T max ) < i, the best response dynamics in (fTTT ) is a contraction with the 

modulus ||T max || ^ mat with respect to the norm max ne ^ ^P— . We can conclude that, the best response 

dynamics has a unique fixed point a* and, given any initial value a , the update sequence {a*} converges 

to the fixed point a*. 

In the sequential update case, the convergence result can be established by using the proposition 1.4 

in EH- The key step is to obtain 

el+i . . r e\ +1 e\ 

max — _ 

neM W n ' \. j<n Wj ' j>n Wj 

A simple induction on n yields 



max -2— < 2p(T max ) • max <^ max -^- , max -± \. (70) 

W n I j<n Wj j>n Wj J 



max %- < 2p(T max ) • max ^L (71) 

neAf W n neAf W n 

for all n. Therefore, inequality (|65l ) also holds for the sequential update and the contraction iteration 
globally converges to a unique equilibrium. ■ 



31 



Appendix B 
Proof of Theorem [2] 

If F^ nn > 0, Vm 7^ n, k, the inequalities after (l63l) become 

iC K 



vn^n fc=l fc=l 

" k,t „k,t-l 



™{EE[M - a - -1 )] .EEl^*-^*" 1 )] } (72) 

m^n fc=l mj^n k=l 

_ K K 

< J2 maxFi • max { ^ [oft* - a^ 1 ] , J] [aft* - a^ 1 ] } (73) 

K 
E m ^ F mn-^2 [ 



(74) 



m^n k=l 

Similarly, for F^ n < 0, Vm 7^ n, k, we have 

max { E E [i(* - ^ H )] > E E [ F -(°™ - «™ _1 )] } ( 75 ) 

mj^n k=l m^n fc=l 

K K 



m^n fc=l fc=l 

A' 



< J] max{-i^„} • max { J] [oft* - a^ 4 x ] , J] [a*>* - o*i* X ] } (76) 

(77) 



,i n k,t-l 



m^n fc=l 

Therefore, if i^ n > 0, Vm 7^ n, fc or F^ n < 0, Vm 7^ n, fc, given (IC2b . the sequence {a^} contracts with 
the modulus p(T max ) < 1 under the norm max ne jv h t ] and the convergence follows readily. ■ 

Appendix C 
Proof of Theorem [3] 

Let || -||2 denote the weighted Euclidean norm with weights w = [w% . . . wk] T , i-e. ||x||J = Q2 l i w i\ x i\ 2 ) 1 ^ 2 
ll33l . Define the simplex 

1 K 
S±{*GK K :-^x k = l,xf n <x k < xf ax , Vfc = 1,2, . . .,k}. (78) 

fc=l 
in which Ylk 3 ^ 3 * — ^- The following lemma is needed to prove Theorem [3] 

Lemma 2: The projection with respect to the weighted Euclidean norm with weights w, of the K- 
dimensional real vector — xo = — [xo,i, • • • , £o,x] T onto the simplex 5 defined in (|78T ). denoted by [— xo]^, 
is the optimal solution to the following convex optimization problem: 

[-x-ols - argmin||x - (-x )||!' (79) 

xG<S 
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and takes the following form: 



X i 



w k 



%0,k 



,k = l,...,K 



(80) 



where A > is chosen in order to satisfy the constraint -^ Ylk=i x k = -*-• 
Proof of Lemma |2} See Corollary 2 in |[20l . ■ 
For /&„(•) defined in (l20l) . user n updates its action according to 



,*fc 



C(a-n,A*)= (--) 



1 Nl- 



-_pfc 



(A*) 






m^n 



k k 
mn^in 



and A* is chosen to satisfy X^fe=i 
and the coupling vector as 



K *k 



(81) 



A „*k 



M n . Define the vector update operator as [BR(a_ n )]fc = a n 

(82) 



[C n (^ n )] k ±^+Y J F 1 



pk 

^ nn 



k k 

mn m 



m^n 



with k € {1, . . . , K}. We also define 

F' ^dias^F 1 F 2 F 

mn ""*& \ ^ mn ' mn ' • • • ' 7 

and 



K 

mn 



a., 



«I "n 



Q 



K 



pi ' 7?2 ' • • • ' pK 

-*■ nn J nn *- nn 



Therefore, the coupling vector can be alternatively rewritten as 

*-'n( a — n) = C^n + / _, ^mn a "i- 
m^n 

Define a weight matrix W = [wi . . . wjv] hi which the element [W]fe n is chosen according to 

(pk 

V- 1 nn 



[W] 



kn 



[V?n]k 



?k \l+i 
nn) 



^K 



2—ik=l\ nn 



\1+: 



(83) 



(84) 



(85) 



(86) 



By Lemma |2j we know that the vector update operator BR n (a_ n ) in (|2TT ) can be interpreted as the 



projection of the coupling vector — C n (a_ n ) onto user n's action set A n with respect to 



', i.e. 



BR n (a_ n ) = [-C n (a_„)] J 



(87) 



Given any aS\aS> E A, we define respectively, for each user n, the weighted Euclidean distances 



between these two vectors and their projected vectors using (1871 ) as e n 



.(2) 



,(1)11*.. 



and e BRn 



|BR n (a_„) — BR n (al„)|L". Again, we first prove the convergence of the parallel update case in (fTTT ). 
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We have Vn € M, 



e BR„ 



[-C n {^ n )Y- - [-C(aW)] 



.(2) 



™ ; J A, 

,i)vir» 



(2) ^ w„ 



< C„(a^)-C n (a 

II IIZ 

= II V F' a( 2 ) -VF' a( r 

/ ,, mn m / 4 mn m 



Vf' 



l a(?)-a(3 ) l 

rrm \ m m I 



rn=£n 



<TW (a' 2 )-a«) = V 

— / j ran \ m m } / 4 



m^n 



E 



m^n 



K 



A J2[Wn}k{[V'mn\kk) (« 
\ fc=l 



2(^(2)k_ (l)fe 



m^n \ k=l 

< J]max(|[F: 



V[w 1 /V 1 i W ^k \ 2 ( (2)k (l)k^ 2 

/ J [W m J fc M r mn J fcfc r _„ i I l°m ~ O m 



[W m jfc 



\kk\ 



[W„Jfc 



^[vfm}k(am - ai\' k j 



(88) 
(89) 

(90) 

(91) 



— V maxYlfF' k, I ^M lln (2) n (1) H Wm 

— / ., ILLdJC UL J mnJfcfe| r l )\\ A m A m || 2 

— V^ fS max l p 

where (l88l) follows from the non-expansion property of the projector [-H" in the norm 

Proposition 3.2(c) in [31]), d90]) follows from the triangle inequality (33), and S max in (HI) is defined 

according to ((22l . 

The rest of the proof is similar as the proof after equation (I65T ) in Appendix A. Details are omitted 
due to space limitations. ■ 



(92) 
(93) 

(94) 
2" (See 
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Appendix D 
Proof of Theorem [4] 

The beginning part of the proof is the same as the proof of Theorem [Q For any user n with general 
/„(•), the inequalities after d59l become 

K 

J2[a k / +1 - a k n Y < mai{^(A*),^)} 



fc=i 



(- X A' 

max £ [fn^-n) - fn^-n)} , E [/»(0 " /»(^ 



fc=l 
A 



fc=l 



(95) 



max 



E[EE|(e.«'-e'f,E[EE|fc)«' 



a *',t-i> 



fc=l m=£nk'=l 



fc=l m^nk'=l 



(96) 



< max 



A ' K <-f)f k i+ K K rf)f k 

EV Y^ •'n ft* W fc',« fe'.t-lNl Y^ Y"^ Y^ °Jn /A \/ k't fc't-l\ 

1^ 1^ [-Q^^-uKa^ -a m ' )\ ,}^1^ 1^ [g^^-n)( a m' -«m' ) 



YdaK 

k=lm^nk'=l 



k' 
k=lm^nk'=l 



(97) 



\dg 



HEEE ^«*-«x#' - ^ _1 )J > E E E [^-)(^ 

lm^nfc'=lfc=l m m ^nfc'=lfc=l m 



r^ 






A 



< , rf 



^E h»E|^ 

m^n I fe=l m 



E[(a^-«^)] + +E[(4; 



,i „fc',t-l> 



fc'=l 

A 



fe'=l 



E 2 - |^E|I§«M} •£[(# 



max 

k 
mj^n \ k=l 



• 4 a^- 1 ) 



fe'=l 



(98) 
(99) 

(100) 



where d95b follows from the definition of p n,t and q n,t and the expression of a n ' and S*(a_ n ,A) 
in (fTTT ) and (l26l) . d96l ) follows from the mean value theorem for vector-valued functions with £ 4 = 
aa* + (1 — a)a 4 ~ 1 and a G [0, 1]. By (IC4b . it is straightforward to show that the iteration is a contraction 
by following the same arguments in Appendix A. The rest of the proof is omitted. ■ 



Appendix E 
Proof of Theorem [6] 

The gradient play algorithm in (|46*1) is in fact a gradient projection algorithm with constant stepsize k. 
In order to establish its convergence, we first need to prove that the gradient of the objective in (l40l) is 
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Lipschitz continuous, with a Lipschitz constant given by L > 0, i.e. 

N N 



V(j^n n (x)) - vE n «bO) 



< L||x-y||, Vx,y G A 



(101) 



n=l 



n=l 



It is known that it has the property of Lipschitz continuity if it has a Hessian bounded in the Euclidean 
norm. 

The Hessian matrix H of En=i M «( a ) can ^ e decomposed into two matrices: H = Hi +H2, in which 
the elements of matrix Hi are 



d 2 



N K 



?k „fc 



/-j /-~i ^n [a n + 2_, ^mn a r, 
n=l k=l m^n 

da^daj 



( N 



E u_ h n f h _i_ v^ rpk n k \ rpk rpk :f h, - 
d 2 x \ n ' 2-*i r mn u m) r mn r lni u "' ~~ J 



n=l 



rn=£n 

0. 



(102) 



otherwise. 



with F* 



1 and the elements of matrix Ho are 



& 1 



N K 



E E <£(a_n) 

71=1 fe = l 



EE 



<9 2 ff»(a-n) 
Recall that g*(-) is Lipschitz continuous and it satisfies 



(103) 



V<?n(x)-VSn(y) 



<Z/||x-y||, Vn,fc,x,yGi_ n . 



Consequently, we have HH2H2 < NKL'. As a result, we can estimate the Lipschitz constant L using the 
following inequalities 

a 2 ^ v v 



|H|| 2 < ||Hi|| 2 + ||H 2 ||2< VllHillillHiHoo + NKL' < sup 

x,n,k 



d 2 x 



■™fT,T,\ F LFl k n\+NKL>. 

m=l n=l 

(104) 
We can choose the RHS of (11041 ) as the Lipschitz constant L . By Proposition 3.4 in IJ3TI , we know 
that if < k < 2/L, the sequence a' generated by the gradient projection algorithm in ( |47T ) and (|48l ) 
converges to a limiting point at which the KKT conditions in (|4"T1)- (|431 are satisfied. ■ 



Appendix F 
Proof of Theorem [7] 

We know from the proof of Theorem [6] that, under Condition (1C71 ). En=i n n( a ) * s Lipschitz continuous 
and the inequality in d 101b holds. Recall that En=i n «( x ) * s continuously differentiable. Therefore, by 
the descent lemma [31], we have 



N 



N 



N 



5^^n(x) > ^«„(y) + (x-y) T - v(^ n «(y)) ~ 7ll x ~y|l2' Vx '^ G • A - 



(105) 



n=l 



n=l 



n=l 
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Therefore, in order to prove Y^ n =l n «( a *) — E n =i u n( at 1 )» we only need to show that 

N 



n=l 

for sufficiently small k. Substituting (l50l into (11061 ). we can see that it is equivalent to 



j: EKV--; 1 ) - atn • a£ "=C-i ^ • f EE Woo - ^) 2 - (107) 

n=l fc=l ^ an re=l fc=l 

By equation d49b , we have 

N 






B' n k (^) - a*/" 1 = {-^}~ (A n + w* - v' n k + J2 ^n 1 ) - E ^ofe*" 1 - "n^ 1 (108) 



and 



^Z/n=l M "( a ) _ &h n / k,t-l i V^ pfc n k ' t ~ 1 ) — V^ 7 r fc ' 4 ~ 1 

_ fc t— 1 Q V u n ' / j r mn u m ) / 4 n mn • 



(109) 



By the mean value theorem, there exists £„ G 7?. such that 

q V u n ~r / j * mn^m ) / j "mn 

dh n ( k,t-l i V^ r?k „k,t-l\ (\ ,,.k ,,'k i V^ ^M-l\ , /\ _i_ „fe '* 

Multiplying (1108b and ( 11091 ) leads to 



E EK fc (^) - tin • Ew :C-i 

71=1 fe=l "°»1 

n=l fe=l I. m^n m^n 



k k,t-l 
mn^m 



N K 

+ E EK'fc 1 ) " a n 1 ' 1 ) ■ ( A « + <- "» ) ( 11Q ) 

n=l fc=l 

In the following, we differentiate two cases in which the Lagrange multipliers \ n ,v k ,v k take different 
values. 
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First of all, if A n = v\ = v' k = for all k, n, equation (II 10b can be simplified as 

N K AX^ N I t-X\ 

EE^-fc 1 ) - tin ■ E ":C-i } 

n=l fc=l aa n 

N K p,2 h k ( ,auk . _i . . N 

= - ££ w(^) • {^} (An + ^ - ,;* + E ^n- 1 ) - a^- 1 - £ *; 



n=l fc=l I. mj^n m^n 

(111) 

On the other hand, if \ n > 0, u k > 0, or v' k > for some k, n. Due to complementary slackness in 
(1421) and (|33i We know that 

A n > => £ itfOO = M n > J] a^" 1 , 
fc=i fe=i 

^n > U => #n ( a -n J = a n,k > a n , 
,,'fc \ n ^ u'kf„t— 1\ ,min ^- „k,t— 1 

i/„ > U, =>• #„ (a_ n ) - o„ jfe < o n ' . 

As a result, the last term in (II 10b satisfy 

/Y K N K 

EEK'fc 1 ) - a « 4_1 ) • ( A « + ^ - ^ fc ) = E A « £(*» oO - a « i_1 )+ 

n=l fc=l n=l fc=l 

AT # TV K 

£ E ^ W(0 - ^n'*- 1 ) + E E W" 1 - ^n (atn 1 )) > 0. (112) 

n=l fc=l n=l fc=l 

Therefore, in both cases, the following inequality holds 



iV K A\^ N ( 



oa n ' 



n=lfc=l 

x a2uk^\ ( , auk . _i , . ^ 

7fc _fc.£-l 



^-Z^Z^ S ^P Q2 X ' (-^J (*» + "n - ^n + 2^ f »m J ~ a n ~ 2^ K 

n=l fe=l 1. mjtn m^n 

N K pfiukf \ 

= " E E -P -^ ■ ( B n(*-n) ~ af/" 1 ) 2 - (1 13) 

n=lk=l X 

\ v 9 / d 2 h k (x)\ 

Finally, we can conclude that the inequality in (11071) holds for k < ^ • (— max n ^ sup x — ^ ' ). Recall 
that Jacobi update requires k € (0,1]. The stepsize k can be eventually chosen as < k < min{|; • 
(- max n)fc sup^. — Q%f-t),l\ U 

Appendix G 
Upper bound of p{S k ) 

Denote 1 T = [1 1 • • • if. If we fix 1 - r^ for Vm,k, we have l T R fc = (1 - r^ )l T . Note that 
T fe = (I - R fc )- X = I + £ 4 =i( Rfe ) i - We have lTTfc = 1 T (I + E^i^)*) = 1 T + (1 - ^ )l T T fc and 
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l T T fc = I3^1 T Therefore, |T*|i = ^-. Since F k m = ^^ and T k = I + ^(R*)*, we know 
[Y fc ] nn > 1 for Vn. Denote a diagonal matrix diag(T fc ) with the entries of T fc on the diagonal. Recall 
that [S k ] mn = F^ n for m ^ n, and [S k } nn = for n € N. We can conclude that p(S k ) < (S^U < 
\(T k ) T - diag(T fc )U < |(T fe fU - 1 = |T fc |! - l = -J- - l. 
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